Diiusion of Context and Credit Information in Markovian Models

This paper studies the problem of ergodicity of transition probabilitymatrices in Marko vian models such as hidden Markov models HMMs and how it makes very di cult the task of learning to represent long term context for sequential data This phenomenon hurts the forward propagation of long term context information as well as learning a hidden state representation to represent long term context which depends on propagating credit information backwards in time Using results from Markov chain theory we show that this problem of di usion of context and credit is reduced when the transition probabilities approach or i e the transition probability matrices are sparse and the model essen tially deterministic The results found in this paper apply to learning approaches based on continuous optimization such as gradient descent and the Baum Welch algorithm Introduction Problems of learning on temporal domains can be signi cantly hindered by the presence of long term dependencies in the training data A sequence of random variables e g a sequence of observations fy y yt yT g denoted y T is said to exhibit long term dependencies if the variables yt at a given time t are signi cantly dependent on the variables yt at much earlier times t t In these cases a system trained on this data e g to model its distribution or make classi cations or predictions has to be able to store for arbitrarily long durations bits of information in its state variable called xt here In general the di culty is not only to represent these long term dependencies but also to learn a representation of past context which takes them into account Recurrent neural networks Rumelhart Hinton Williams Williams Zipser for example have an internal state and a rich expressive power that provide them with the necessary long term memory capabilities Algorithms that could e ciently learn to represent long term context would be useful in many areas of Arti cial Intelligence For example they could be applied to many problems in natural language processing both at the symbolic level e g learning grammars and language models and subsymbolic level e g modeling prosody for speech recognition or synthesis In order to train the learning system however an e ective mechanism of credit assign ment through time is needed To change the parameters of the system in order to change the internal state of the system at time t so as to improve the internal state of the system c AI Access Foundation and Morgan Kaufmann Publishers All rights reserved